/LMAR=0/XLINE=3/FONT#0=BAXL30/FONT#1=BAXI30/FONT#2=BAXB30/FONT#3=BAXS30/FONT#4=GRFX25/FONT#9=MS25















␈↓ ↓H␈↓␈↓ ¬|Supplement


␈↓ ↓H␈↓␈↓ ε5for


␈↓ ↓H␈↓α␈↓ ∧jLISP Timing Evaluation Project






␈↓ ↓H␈↓␈↓ ¬;Principal Investigator

␈↓ ↓H␈↓␈↓ ¬aJohn McCarthy




␈↓ ↓H␈↓␈↓ ¬;Associate Investigator

␈↓ ↓H␈↓␈↓ ¬]Richard Gabriel
␈↓ ↓H␈↓α␈↓ <1


␈↓ ↓H␈↓α␈↓ ∧V1.  LISP Timing Evaluation Project

␈↓ ↓H␈↓␈↓αPersonnel: ␈↓ John McCarthy, Richard Gabriel ␈↓↓Student Research Assistants:␈↓ ␈↓↓␈↓βE␈↓↓␈↓-

␈↓ ↓H␈↓The Lisp Timing Evaluation Project began at the end of February and is slowly taking shape.
␈↓ ↓H␈↓Originally simply a timing project, other evaluations are being considered.

␈↓ ↓H␈↓The thrust of the project is to provide both objective and subjective bases for rational choice
␈↓ ↓H␈↓among the various Lisps and Lisp systems available today or to be available in the near future.
␈↓ ↓H␈↓The objective measures are to be provided through the use of benchmarks which will be run on
␈↓ ↓H␈↓the various systems in the test with measurements made in terms of CPU time. These benchmarks
␈↓ ↓H␈↓will be/are being provided by people at the various sites in order to provide a range of interesting
␈↓ ↓H␈↓benchmarks, not merely a few arti␈↓βC␈↓cial ones. This latter point is important for several reasons.
␈↓ ↓H␈↓First, it will test the various systems with the range of real applications that are going on. Second,
␈↓ ↓H␈↓it will bring too light the esoteric features used by research programmers (and, hence, shed some
␈↓ ↓H␈↓light on Hearn's claim that a relatively small subset of Lisp will su␈↓β@␈↓ice for most research
␈↓ ↓H␈↓programming in Lisp).

␈↓ ↓H␈↓In the event that benchmarks covering the range of desired features are not available, `arti␈↓βC␈↓cial'
␈↓ ↓H␈↓benchmarks will be supplied. Either benchmarks will be solicited from people and systems that are
␈↓ ↓H␈↓known to perform such operations or they will be written explicitly for the situation.

␈↓ ↓H␈↓The subjective measures are descriptions of the systems provided by the volunteers along with
␈↓ ↓H␈↓experiences with the translating the various benchmarks: since the benchmarks are not restricted
␈↓ ↓H␈↓in any way a translation phase for each site is required. The tools and problems associated with
␈↓ ↓H␈↓each translation (with the original and translated programs as evidence) can be interpreted as a
␈↓ ↓H␈↓measure of the expressive power of a given Lisp/Lisp system.

␈↓ ↓H␈↓A ␈↓βC␈↓nal measure of the non-language e␈↓β@␈↓iciency will be attempted, though there will be some
␈↓ ↓H␈↓technical problems here. For example, garbage collection, system paging overhead, and external
␈↓ ↓H␈↓loads on the machines a␈↓β@␈↓ect the `usability' of a Lisp without being directly associated with the
␈↓ ↓H␈↓language.

␈↓ ↓H␈↓The following is a list of the systems to be tested as known at this point:

␈↓ ↓H␈↓        InterLisp on MAX, Dolphin, Dorado
␈↓ ↓H␈↓        MacLisp on SAIL
␈↓ ↓H␈↓        NIL on S-1
␈↓ ↓H␈↓        InterLisp on SUMEX
␈↓ ↓H␈↓        UCILisp on Rutgers
␈↓ ↓H␈↓        SpiceLisp on PERQ
␈↓ ↓H␈↓        Lisp Machine (Symbolics, CADR)
␈↓ ↓H␈↓        MacLisp on AI, MC, NIL on VAX
␈↓ ↓H␈↓        InterLisp on F2
␈↓ ↓H␈↓        Standard Lisp on TOPS-10, B-1700 
␈↓ ↓H␈↓        Lisp370
␈↓ ↓H␈↓        TLC-Lisp on Z-80
␈↓ ↓H␈↓        muLisp on Z-80
␈↓ ↓H␈↓        Muddle on DMS
␈↓ ↓H␈↓        Rutgers Lisp
␈↓ ↓H␈↓        Multics MacLisp
␈↓ ↓H␈↓αLISP Timing Evaluation Project␈↓ :2


␈↓ ↓H␈↓        Jericho InterLisp
␈↓ ↓H␈↓        Cromemco Lisp on Z80
␈↓ ↓H␈↓        Franz Lisp on VAX UNIX
␈↓ ↓H␈↓        UTILisp


␈↓ ↓H␈↓Some proposed features of Lisps that will be tested, either by naturally occurring benchmarks or
␈↓ ↓H␈↓by arti␈↓βC␈↓cial ones are:

␈↓ ↓H␈↓        Array reference and storage (random access)
␈↓ ↓H␈↓        Array reference and storage (matrix access)
␈↓ ↓H␈↓        Array reference and storage (matrix inversion)
␈↓ ↓H␈↓        Short list structure (records, hunks...)
␈↓ ↓H␈↓        Long list structure (cdr access)
␈↓ ↓H␈↓        CAR heavy structures
␈↓ ↓H␈↓        CDR heavy structures
␈↓ ↓H␈↓        Interpreted function calls
␈↓ ↓H␈↓        Compiled function calls
␈↓ ↓H␈↓        Smashed function calls
␈↓ ↓H␈↓        Table function calls (FUNCALL, SUBRCALL)
␈↓ ↓H␈↓        Tail recursion (?)
␈↓ ↓H␈↓        Block compiling
␈↓ ↓H␈↓        Reader speed
␈↓ ↓H␈↓        Property list structures
␈↓ ↓H␈↓        Atom structures (saturated obarrays)
␈↓ ↓H␈↓        Internal loops
␈↓ ↓H␈↓        Trigonometric functions
␈↓ ↓H␈↓        Arithmetic (floating and fixed)
␈↓ ↓H␈↓        Special variable lookup
␈↓ ↓H␈↓        Local variable lookup
␈↓ ↓H␈↓        CONS time
␈↓ ↓H␈↓        GC time
␈↓ ↓H␈↓        Compiled code load time
␈↓ ↓H␈↓        EQ vs EQUAL test time
␈↓ ↓H␈↓        Arithmetic predicates
␈↓ ↓H␈↓        Type determination


␈↓ ↓H␈↓The remaining work is to complete the gathering and initial evaluation of the benchmarks,
␈↓ ↓H␈↓collecting results from the various systems, collection of system descriptions, factoring out Lisp
␈↓ ↓H␈↓versus system overhead from raw data, evaluation of the impact of system overhead and garbage
␈↓ ↓H␈↓collection, and publication and distribution of results. Since some machines and systems are not yet
␈↓ ↓H␈↓available for testing, the project will be completed by June 1982.